Real-valued All-Dimensions Search: Low-overhead Rapid Searching over Subsets of Attributes

This paper is about searching the combina­ torial space of contingency tables during the inner loop of a nonlinear statistical optimiza­ tion. Examples of this operation in various data analytic communities include search­ ing for nonlinear combinations of attributes that contribute significantly to a regression (Statistics), searching for items to include in a decision list (machine learning) and associ­ ation rule hunting (Data Mining). This paper investigates a new, efficient ap­ proach to this class of problems, called RAD­ SEARCH (Real-valued All-Dimensions-tree Search). RADSEARCH finds the global op­ timum, and this gives us the opportunity to empirically evaluate the question: apart from algorithmic elegance what does this attention to optimality buy us? We compare RADSEARCH with other recent successful search algorithms such as CN2, PRIM, APriori, OPUS and DenseMiner. F i­ nally, we introduce RADREG, a new regres­ sion algorithm for learning real-valued out­ puts based on RADSEARCHing for high­ order interactions.